THE STRUCTURE OF THE NORMALISERS OF THE 
CONGRUENCE SUBGROUPS OF THE HECKE GROUP G5 



Mong Lung Lang 

Abstract. Let A = 2cos(7r/5) and let t G Z[A]. Denote the normaliser of 
Go(t) of the Hecke group G5 in PSL2(R) by N{Go{t)). We prove that 
N{Go{t)) = Go(t//i), where h is the largest divisor of 4 such that di- 
vides T. Further, Ar(Go(r))/Go(T) is either 1 (if h = 1), Z2 X Z2 (if /i = 2) or 
Z4 X Z4 (if h = 4). 

1. Introduction 

In [LT], it is shown that Go(t) C G5 is self- normalised if r is square free (G5 is 
the Hecke group associated to A = 2cos (tt/S), an element of Gs is in Go(t) if and 
only if its (2,l)-entry is a multiple of r). In this article, we complete our study 
of the normaliser of Go(t). This is the first step towards the determination of 
the normalisers of the congruence subgroups of all Hecke groups Gq. Note that 
the normalisers of the congruence subgroups of G3 = PSL2{'Z), G4, Gq have been 
determined by Akbas-Singerman [AS], Atkin-Lehner [AL], Conway [C], and Lang 
[La]. Our main result is (Section 6) : 

The Main Theorem. Let (r) be a nontrivial ideal of I^IX] and let N{Go{t)) be 
the normaliser of Go{t) in PS'L2(K). Then N{Go{t)) — Go{T/h), where h is the 
largest divisor of =4 such that h^ divides r. Further, N {Go{t)) / Go{t) is either 
1 {ifh = 1), Z2 X Z2 {ifh = 2) or Z4 x Z4 {if h = 4). 

It is worthwhile to point out that as an application of our main theorem, the number 
4 in the main theorem enjoys the following property (see Appendix A) : 

Proposition A2. Ifr £ Z[A] is G^-elementary (see the following definition), then 
r is a divisor of h^ = A. 

Definition, x/y £ Q [A] is in reduced form if ^ ^ ^ is a column vector of some 

CT € G5. An element r E Z[A] is called Gs-elementary if r satisfies the following 
property : Whenever x and y are chosen such that x/ry is in reduced form, x must 
admit the property that x^ = 1 (mod r). 

It is known that the h satisfies the above proposition for Gg — PSL2 (Z) is — 24 
(Conway [C]) and that the number 24 plays a very important role in the description 
of the normaUsers of congruence subgroups of G3, G4 and Gg ([C], [CN], [La]). We 
suspect that the hq for the ring associated to the set of finite cusps of Gq plays some 
role in the determination of the normalisers and hope that such possibility will be 
considered in the future. We do not know whether Proposition A2 can be proved 
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without the usage of the main theorem. A weaker version of this proposition and 
its direct proof can be found in Appendix A (Proposition A4). 

The remaining of this article is organised as follows. In section 3, we give a brief 
introduction of the reduced forms which a great detail can be found in [R]. We 
actually determine the reduced forms of certain members in Q [A] which are useful 
in the determination of the normalisers of Go{t). Sections 4 and 5 give supergroups 
and subgroups of N{Go{t)). Section 6 is devoted to the proof of the main theorem. 

2. Known results 

Theorem 2.1. (Theorem 8 of [LT]) Let (r) be a nontrivial ideal o/Z[A] and let 
(tq) he the maximal ideal of T^}^ such that (tq) C (r). Then N{Gq{t)) C Go(to). 

Note that if t = n^'r* where the piS are primes, then tq = ]^p™'*~'-™»/2j ^ ^j^g^.^ 
\_x\ is the largest integer less than or equal to x. Throughout this article, we let 

r 1 TT mi—\mi/2\ 

It is clear that if a and b are coprime to each other, then [ab] = [a] [6] . 
Lemma 2.2. (Lemma 1 of [CLLT]) Let A be a non-zero ideal o/Z[A]. Then 

[G^:Go{A)]=N{A)\{{l + N{P)-'), 

P\A 

where the product is over the set of all prime ideals P which divide A and N{A) is 
the absolute norm of A. 

3. Reduced forms 

Let x,y E Z[A]. x/y is in reduced form if ^ ^ ^ is a column vector of some 

a G G5. The purpose this section is to determine the reduced forms of 3/nA, 9/nA 
and 5/nA which will be used in the proof of Propositions 4.3 and 4.4. 

For any a,b € Z[A] such that the greatest common divisor of a and b is a unit, 
applying results of Leutbecher ([L1],[L2]), there exists a unique rational integer 
e{a/b) such that 

is in reduced form. We shall called e{a/b) the reduced factor of a/6. Since the 
reduced forms play a central role in the determination of the normaliser (see Lemma 
4.1), we shall now give an algorithm that enables us to determine the reduced factor. 

Let a,b G Z[A] \ {0} be given such that the greatest common divisor of a and b is 
a unit. Then there exists a unique rational integer q such that 

(i) a = {qX)b + r, 

(ii) -|6A|/2 < |r| < |6A|/2. 

We call such a division algorithm pseudo-Euclidean (see [R] for more details). In 

terms of matrices, the above can be written as 



NORMALISERS OF CONGRUENCE SUBGROUPS 3 

Note that ^ ^ '^^^ G G. Applying the pseudo Euchdean algorithm repeatedly, 
one has, 

a = {qiX)b + ri , 
b = (g2A)ri + r2 , 
ri = ((?2A)r2 + rs , 



rk = (9fe+iA)rfc+i + rfe+2 , 

rfe+i = {qk+2>^)rk+2 + . 
The finiteness of the algorithm is governed by the fact that the set of cusps of G is 
Q [A] U {oo}. Note that in terms of matrices, the above can be written as 




where A G Gs. It is clear that gcd(a, &) = gcd(6, ri) = ••• = gcd(ri;+i, rfe+2) = 
rk+2 is a unit (a and b are coprime). As 

(i) |rfe+2| < 1, 

(ii) A is a primitive unit. 

there exists e{a/b) G N U {0} such that |rfe+2| = X-<°'/^'> . Multiplying (3.1) by 
X<«-/b)^ one has 

[ bx^-/") ) ^ [ ) ■ 

Since A G G5 and ±1/0 is a reduced form, we conclude that aA^("/''V6A«("/*') is the 
reduced form of a/b and e{a/b) is the reduced factor of a/b. Note that it is clear 
from the above argument that 

e{a/b) = e{ri/b) = ■■■ = e{rk+i/rk+2)- (3.2) 

The following lemma shows that ii m — n ~ kr, r E Z, then e{k/nX) = e{k/mX). 
Note that this equality does not hold for e{k/n) and e{k/m). 

Lemma 3.1. Let x, u, v G Z[A]^. Suppose that gcd{x,u) is a unit and that 
{u — v)X = xmX for some m G Z. Then e{x/uX) = e{x/vX). 

Proof. Let 

uX = x{qiX) + ri , 
vX = x{q2X) + r2 , 

where qi, 52 € Z, — |a;|A/2 < ri, r2 < |a:|A/2. Since {u — v)X = xmX, we have ri = 
r2- Applying equation (3.2), we have e{x/uX) = e{r\/x) = e{r2/x) = e{x/vX). □ 

Example 3.2. Applying Lemma 3.1, the following gives the reduced factors of 
p'^/nA, where p < 11 is a prime. These reduced forms will be useful in our study 
of Propositions 4.3 and 4.4. 
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ta^e^p"^ /n\) 



2/nX gcd(n,2) = l 
4/nA gcd (n, 2) = 1 
3/nA gcd (n, 3) = 1 



2 2(A + 1) 

2 4(A + 1) 

3 3(2A + 1) 
3 9(2A + 1) 
9 9(34A + 21) 
6 5(8A + 5) 
6 25(8A + 5) 

12 25(144A + 89) 

6 7(8A + 5) 

6 ll(8A + 5) 



9/nA n EE 1,8 (mod 9) 

9/nA n = 2, 4, 5, 7 (mod 9) 

5/nA gcd (n, 5) = 1 

25/nA n= 1,2,23,24 (mod 25) 

25/nA n = 3, 6, 19, 22 (mod 25) 

7/nA gcd (n, 7) = 1 

11/nA n= 1,10 (mod 11) 



Proof. Suppose that gcd (n, 3) = 1. It follows that n = 3A; ± 1. We have 

nA = 3 • fcA ± A , 
3 = A • A + (2 - A) , 
A = (2 - A) • 3A + (2A - 3) , 
2 - A = (2A - 3) • A + . 

As 2A — 3 = A^"^, the reduced factor is 3 and the reduced form of 3/nA is 3A^/nA^. 
The rest can be calculated similarly. □ 

Example 3.3. e((2A - 1)/12) = e((2A - l)/96) = 6. e((2A - 1)/192) = 18. 

By Example 3.3, the reduced forms of (2A- 1)/12, (2A-l)/96 and (2A- 1)/192 are 
(18A+ll)/12(8A + 5), (18A + ll)/96(8A + 5) and (5778A + 3571)/192(2584A + 1597) 
respectively. The above reduced forms will be used in the remark of Lemma 5.1. 



The main purpose of this section is to prove that N{Gq{t)) C Go{T/h), where h is 
the largest divisor of 4 such that /i^|t (Theorem 4.5). In order to achieve this, we 
need two technical lemmas which can be found in subsection 4.1. The main result 
(Theorem 4.5) is in fact a corollary of Propositions 4.3 and 4.4. 

4.1. Technical Lemmas. The following two lemmas are useful towards the deter- 
mination of a supergroup of N{Go{t)). 

Lemma 4.1. Let t, [r] G Z[A] be given as in Theorem 2.1 and let u, w be chosen 
such that u/wT is in reduced form. Suppose that gcd{T/[T],u'^ — 1) = x. Then 



Proof. Since u/wt is a reduced form, by results of Leutbecher ([LI], [L2]), Go{t) 
has an element of the form 



4. Supergroups of N{Go{t)) 



N{Go{t)) c Go{t/x). 
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Let A = ^ ^ ^ ) ^ -^(^o(t)) . We need to show that A e Gq{t/x). Equivalently, 

we need to show that c is a multiple of t/x. By Theorem 2.1, is a multiple of r. 
Since 

~ ( cdu + w(Pt - (?v — cdz * ) ^ ^o{t), 

we conclude that cdu + wd?T — (?v — cdz = cd{u — z) + wd^r — c^v is a multiple 
of r. It is clear that the last two terms in the expression are multiples of r (c^ is a 
multiple of r). Hence 

T\cd{u — z). 

Since gcd (c, d) = 1 and [r]|c, we have gcd (t, d) = 1. Hence the above is the same 
as t\c{u — z). Since both r and c are multiples of [r], we have 



r 

M 

Hence 



^{u-z). (4.1) 



t/\t\ 

cI[t] is a multiple of — -- — 77^; r- . (4.2) 

gcd {t/[t\,u-z) 

We shall now determine gcd (r/[r], u — z). Note first that gcd {u, r) = 1. It follows 
that 

gcd (t/[t], u — z) = gcd (t/[t], ti{ti — z)). 

Since uz — vut = 1 (this is the determinant of a), the right hand side of the above 
equation becomes 

gcd{T/[T],u{u — z)) = gcd{T/[T],u^ — 1 — vwt) = gcd{T/[T],u'^ — 1) = x. 
Statement (4.2) now can be rephrased as 

t/[t1 

c/[t] is a multiple of . 

X 

Therefore, c is a multiple of t/x. This completes the proof of the lemma. □ 
Definition. (Principal Congruence Subgroups). Let 

Then G{t) is a normal subgroup of G5. G{t) is called the principal congruence 
subgroup of G5 . 

Lemma 4.2. Let V = (Go(3"V), G{v)) , where gcd{3, v) = 1. Then V = Gq{v). 

Proof. Since there is nothing to prove if m = 0, we shall assume that m > 1. 
By Lemma 2.2, we have [Gn{v) : Go(3™z/)] = S^'^-^xq. In order to show that 
V = Go(i^), it suffices to show that V possesses more than [Go(i^) : Go(3'"!^)]/2 = 
32m-25 i^f^ Go(3'"j^)-cosets. Let 

"={^-=(j f )(,4a)A ;) :0<.,.<3--l,,cd(3,,) = l}. 
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It is clear that ft CV and that V possesses 2 • s^*"-! elements (2 • 3"*"-^ choices of 
y and 3™ choices of x). Applying the fact that = A + 1, one can show by direct 
calculation that 

^,^Go(3"z/) = A^.y^GoiS'-^iy) iS y = y', y^{x -x') = {mod 3™). 

Consequently, members in ft give different Go(3™J')-cosets. Since \Q\ = 2 • 3^™""^, 
we conclude that V has at least 2 • 3^"^-'^ > 3^"'-'^5 left Go(3'"z/)-cosets. This 
completes the proof of the lemma. □ 

4.2. Supergroups of N{Gq(t)). In this section, we apply our results in section 4.1 
and our knowledge about the reduced forms of 3/nA, 9/nX and 5/nA to give some 
more accurate description of the a;'s in Lemma 4.1. This is done in Proposition 4.3 
and 4.4. As a corollary of these two propositions, we can show that Goir/h) (see 
Theorem 4.5) is a supergroup for N{Go(t)). 

Proposition 4.3. Let t = 3"^p (3 is a prime in Z[A]), where gcd{3,v) = 1. Then 
N{Go{t)) C N{Go{u)) C Go(z//gcd(z//M,4). 

Proof. We shall first prove that N{Go{t)) C N{Go{iy)) : Let a; e iV(Go(3"i^)). We 
need show show that x € N{Go{u)). By Theorem 2.1, x € Go([3'"j/]) C G5. Since 
the principal congruence subgroup G{u) is normal in G5, we have xG{u)x~^ = G{v). 
It follows that 

V = (GolS^t.), G(i.)) C xGo{v)x-^ n Go{v) C Go{u). (4.3) 

By Lemma 4.2, V = Go{v). Expression (4.3) now implies that a;Go(a)a;~^nGo(a) = 
Gq{v). This implies that x e N{Gq{v)). Hence 

N{Gq{t)) C N{GM). 

We shall now prove that N{Ga{v)) C Gq{v/ gcd(z^/[i^], 4) : Let n be the smallest 
positive rational integer in [v). Then gcd (n, 3) = 1. By Example 3.2, the reduced 
form of 3/nA is 3A^/nA*. By Lemma 4.1, we have 

N{Gq{v)) C Go{v/x) , where x = gcd {vl\v\, {3\^f - 1). (4.4) 

By Example 3.2, the reduced form of 9/nA is either 9A^/nA'' (if n = 1,8 (mod 9) 
or 9AVnAi° (if n = 2, 4, 5, 7 (mod 9). 

(i) Suppose that n = 1,8 (mod 9). By Lemma 4.1, we have NiG^iy)) C GQ{v/y), 
where 

y = gcd{v/[v],{Q\^f -1). 

Applying (4.4), we have N{Gfi{v)) C Gaiv/y) n Go{v/x). Since Gq{u) fl Go{v) = 
Go(lcm (u, v)) and gcd {{3X^f - 1, (9A-^)2 - 1) = 4, we have 

N{Go{v)) C Go{i^/x)nGo{i^/y) = Go{i^/gcd{i^/H4)). 

(ii) Suppose that n = 2,4,5,7 (mod 9). By Lemma 4.1, we have N{Go{i^j) C 
Go{i'/z), where 

z = gcdiiy/[iyi {9Xy-l). 

Applying (4.4), we have N{Go{v)) Q Go{v/z) n Go{v/x). Since Go{u) n Go{v) = 
Go(lcm (u, v)) and gcd {{3\^f - 1, {%\^f - 1) = 4, we have 

N{Go{u)) C Go{u/z) n Go{u/x) = Go{y/gcd{y/[ulA)). □ 
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Similar to Proposition 4.3, we have the fohowing proposition for the decomposition 
r = V (note that the reduced form of 5/nA is 5A^/nA^ (Example 3.2) and that 
(5A*5)2 - 1 = 16(225A + 139), where the absolute norm of 225A + 139 is 29). 

Proposition 4.4. Let r = \/5 "^v, where gcd{^/5, z^) = 1 (5 is not a prime). Then 

N{Go{t)) C N{Go{iy)) C Go(i^/gcd(i./M, 16(225A + 139)). 

Theorem 4.5. Let (t) be a nontrivial ideal o/Z[A] and let N{Go{t)) be the nor- 
maliser of Go{t) in PS'L2(M). Then N{Go{t)) C Go(r//i), where h is the largest 

divisor of 4 such that h^ is a divisor of t . 

Proof. Let r = 2''3''-\/5 '^a, where gcd(a, 6-\/5) = 1. Applying Propositions 4.3 and 
4.4, we have 



Since [xy] = [x\[y] whenever gcd(a;,y) = 1, 225A + 139 has absolute norm 29 and 
Gq{x) n Go{y) = Go (1cm {x,y)), the above can be simplified into 

N{Go{t)) C Go I J =Go( , ^, , . (4.5) 

\^5cd(2«-L«J,4)y VflC(i(2«-LaJ,4)y ^ ^ 

Note that gcd (2"^LaJ _ 4) equation (4.5) is nothing but the largest divisor /i of 4 
such that h'^\T. This completes the proof of the theorem. □ 

5. Subgroups of 7V(Go(r)) 

The aim of this section is to find large subgroups of iV(Go(r)). It is proved in 
Lemma 5.2 that the supergroups for N{Go{t)) (Theorem 4.5) are actually the 
subgroups of N{Go{t)) as well. 

Recall that Z[A] is a principal ideal domain and 2 is a prime in Z[A]. An element r 
in Z[A] is even if t = 2x for some x G Z[A]. Unlike the rational integers Z, the sum 
of two odd integers in Z[A] docs not have to be even (both 1 and A are odd, yet 
the sum of 1 and A is not even). As a consequence, the trace of a G PSL2{'Z[X\), 
where the (2,1) entry of a is even, does not have to be even (see remark of lemma 
5.1). However, as the following lemma shows, the trace of cr € G5 is even if the 
(2,1) entry of a is even. 

Lemma 5.1. Let ^ 2c ^ ) ^ Go{2). Then a + d, a — d are even and — 1 is a 
multiple of 4. 

Proof. An easy study of a fundamental domain of Go (2) shows that Go(2) can be 
generated by 

1 X\ f 2X+1 -A-2\ / 2A + 1 -A 
1 j 'i^ 2A + 2 -2A-1 J '\ 2A -1 

(see [LLT]). Note that 

(a) the sum and difference between the (1,1) and (2,2) entries of the matrices 
above are even, (b) (2A + 1)^ — 1 is a multiple of 4. 



8 



NORMALISERS OF CONGRUENCE SUBGROUPS 



As a consequence, the generators of Go{2) satisfy the conclusion of our lemma. 
Since every element of Go (2) can be written as a word of the generators of length 
n for some n, the lemma can be proved by applying induction on n. □ 

Remark, (a) An easy study ^ ^''^ 2A A ) ^'^^^^ ^^^^ Lemma 5.1 does not 
hold for PSL2(Z[X]). 

(b) The squares of the (1,1) entries of the following matrices (see Example 3.3) 
are not congruent to 1 modulo 8. This means that the second part of Lemma 5.1 
(a^ — 1 = (mod 4)) can not be improved for the following groups. 



/ 18A+11 *\p^.pfiA { 5778A + 3571 * \ ^ ^ .^qo^ 
1, 96(8A + 5) * ) ^^o(96), i92(2584A+1597) * ) « ^0^192). 

Lemma 5.2. Let r G Z[A]. Suppose that t is even. Then the following holds. 

(a) //4|t, then Go(t/2) C iV(Go(T)). 

(b) 7/16|t, i/ienGo(r/4) CN{Go{t)). 

Proof, (a) hetA=( J \ e Go(r/2), B ^ ( Me Go(r). Then 



ABA- 



* 



cd{u - x)t/2 - c^i>T^/4 - 



Since u — a; is even (Lemma 5.1), t is a multiple of 4 (our assumption), we conclude 
that the three terms in cd{u — x)t/2 — c^vt"^ /A + wd'^T are multiples of r. Hence 
cd{u— x)t / 2 — VT^ / A+wd^ T is a multiple of r. As a consequence, ABA~^ S Go(t). 
This implies that Go(r/2) C N{Gq{t)) . 

^ = ( cr/4 d ) ^ ^ = ( J. l)^ ^«(-)- Tl^- 



ABA- 

Since 



Cd{u - x)t/A - C^VT^ /IQ + WKTT * 



(i) u — X = {v? — 1 — vwt)/u (det B = ux — wvt = 1), 

(ii) — 1 is a multiple of 4 (Lemma 5.1), 

(iii) T is a multiple of 16 (our assumption), 

we conclude that u — a; is a multiple of 4. Hence all three terms in cd{u — x)t/A — 
c^wT^/16 + wd^T are multiples of r. This implies that cd{u — x)t/A — c^fr^/16 + 
wd'^T is a multiple of r. As a consequence, ABA^^ e Go(t). Thus Go(r/4) C 
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6. The Main Theorem : Normaliser of Go{t) 

The following theorem follows immediately by applying Theorem 4.5 and Lemma 
5.2. 

Theorem 6.1. Let (r) be a nontrivial ideal ofZ[X] and let N{Go{t)) be the nor- 
maliser o/Go(r) in PSL2{M.). Then N{Go{t)) = Go{T/h), where h is the largest 

divisor of 4 such that h^ is a divisor of t . 

The following proposition determines the group structure of N(Go{t))/Go{t). 

Proposition 6.2. N {Go{t)) / Gq{t) = Gq{t / h) / Gq{t) is either 1 {if h = 1), Z2 x 
Z2 {if h = 2) or ZiX Z4 {if /i = 4). 

Proof. Recall first that /i^jr. There is nothing to prove for the case h = 1. In 
the case h ^ 2, [N{Go{t)) : Go{t)] = [Go(t/2) : Go(t)] = 4 (Lemma 2.2). For 

any a G Go(t/2), since t/2 is even, we have = ^ 2c |; ^ — ^o(2). 

Applying Lemma 5.1, the (2,l)-entry of cr^, which is 2c(o + d), is a multiple of 
r. Hence ct^ e Go(t). In particular, every nontrivial element in Go{t/2)/Go{t) is 
of order 2. Hence AT (Go (r)) /Go (r) = Go{t/2)/Go{t) ^ Z2 x Z2. We shall now 

assume that h = A. 

(i) [iV(Go(T)) : Go(t)] = [Go(t/4) : Go(r)] = 16 and Go(t/4)/Go(t) is abelian. 
Proof o/(i). By Lemma 2.2, [Go(t/4) : Go(t)] = 16. Let 

ai 61 \ [ 02 &2 

0-1 = „ J , 0-2 = 



Cir/4 di J' ^ \ C2r/4 ^2 
The (2,l)-entry of cricr2crf^cr^^ is 

7" 

— [(a2Ci + diC2){c2bir/4: + did2) - (aiC2 + c?2Ci)(ci62t/4 + ^1^2)]- (6.1) 

Since 16 divides t, 

(6.1) = -^did2{{a2 - d2)ci + 02(^1 - ai)) (mod r). (6.2) 

Applying Lemma 5.1, = 1 ( mod 4). Since Oidi = 1 (mod 4), we conclude that 
a, — dj = (mod 4). Hence the right hand side of (6.2) is a multiple of r. Hence 
(6.1) = (mod r). Equivalently, the (2,l)-entry of c7i(T2CTf ^a-g ^ is a multiple of r. 
Therefore, a-icr2cr^V^^ e Go(t) for all ai,a2 e Gq{t/A). Hence Gq{t / A) / Go{t) is 
abelian. 

(ii) Go(t/4)/Go(t) ^ Z4 X Z4. Proof of (ii). Let 



a = 



a b 
ct/A d 



Applying Lemma 5.1, the (2,l)-entry of is a multiple of r. It follows that 
cr'* e Go(r). Hence every element in Go{t/A)/Gq{t) is of order 1, 2 or 4. Let 

n/4 be the smallest positive rational integer in (t/4). The order of ^ ^^^^4 \ ^ 
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is 4 in Go(t/4)/G'o(t). Since Gq{t/A)/Go{t) is abelian and every element in 
Go(t/4)/G'o(t) is of order 1, 2 or 4, wc have 

Go(t/4)/Go(t) = Z4 X Z4 or Go(r/4)/Go(T) = Z4 x Z2 x Z2. 

Suppose that we are in the latter case. It follows that a representative a of some 
nonidentity element in Go(t/4)/Go(t/2) ^ Z2 x Z2 is of order 2 in Go(r/4)/Go(T). 

Let <y = ( ^ ) be such an element. Since a is of order 2 in Gq{t/A)/Go{t), 



ct/A d 

Then the (2,l)-cntry of o"^ must be a multiple of r. Direct calculation shows that 
the (2,l)-entry of cr^ is c(a + (i)r/4. Since c is odd (this follows from the fact that 
(J ^ Go(r/2)), we have a + d is a multiple of 4. Since ad=l (mod 4) (determinant 
of a is 1), we have = —1 (mod 4). A contradiction (see Lemma 5.1). Hence such 
a does not exist. It follows that Go(t/4)/Go(t) = Z4 x Z4. □ 

Remark. Unlike the modular group case, where ro(n//i) C A^(ro(n)) [h is the 
largest divisor of 24 such that h? divides n), the number 3 does not play any role 
in the normaliser of Go(t) C G5. This can be seen in Theorem 6.1 or the following 

example, which shows that A^(Go(9)) does not contain ^ ^ 



3A 1 

Example. By Example 3.2, the reduced form of 4/9A is 4A^/9A''. It follows that 
Go (9) has an element of the form = ^ g^s ^ ^ • Direct calculation shows that 

3A 1 j 1^ b ) \ -3A 1 j ^ V 21A^ - QaA^ - 36A * 

Since the determinant of a is 1, 46A^ = 1 (mod 9). Hence 21A'^ — 9aA^ — 36A is not 
a multiple of 9. In particular, ^ ^ ^ does not normalise Go (9). 

Appendix A 

Definition Al. An element r G Z[A] is called Gs-elementary if r satisfies the 
following property : Whenever x and y are chosen such that x/ry is in reduced 
form, X must admit the property that = 1 (mod r). 

Proposition A2. If r € Z[A] is G<i- elementary, then r is a divisor of = 4. 



Proof. Let r be elementary and let A = ^ ^ ^ ^ G Go(r), i? = ^ 
Go(r2). Then 



Af!A~^ — 

cdyu — x)r — c vr + wd r * 

Since ux = 1 (mod r^) and r is elementary, wc have u — x = (mod r). Hence 
cd{u — x)r — c? vr"^ +wd'^r'^ is a multiple of r^. As a consequence, ABA^^ E Go(r^). 
This implies that Go(r) = Go(rVr) C N{Goir^)). By Theorem 6.1, N{Gair^)) = 
Go{r^/h), where h is the largest divisor of 4 such that /i^|r^. It follows that Go(r) C 
Go(r^//i). This implies that r is a divisor of 4. □ 
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Discussion. Wc do not have a direct proof of Proposition A2. The foUowing 
weaker version about the reduced forms of G5 does enjoy a proof without the usage 
of the main theorem. 

Definition A3, r € Z[A] is called strongly Gs-elementary if every divisor of r 
is Gs-elementary. 

Proposition A4. Suppose that r is strongly G 5- elementary. Then r is a divisor 
of /15 = 4. 

Proof. Suppose that r has a non-unit odd divisor n. Let N{ti) = riSi = n e Z. 
By Example 3.2, the reduced form of 2/riSi is 2A^/nA^. By our assumption, 

(2A2)2 = (2(A + l)f = 12A + 8 = 1 (mod n). 

Hence ri — 12A + 7 (12A + 7 is a prime of norm 11). By Example 3.2, the reduced 
form of 3/riSiA is SA^/llA"^. Direct calculation shows that (3A^)^ ^ 1 (mod r\). 
A contradiction. Hence r must be a power of 2. Let r = 2™. Applying Lemma 
5.1, r = 2^ admits the above mentioned property. Since (2A + 1)/2A is in reduced 
form (see Lemma 5.1) and (2A + 1)^ ^ 1 (mod 2^), 8 is not Gs-elementary. Hence 
r must be a divisor of h^ = A. □ 

References 

[AL] A. O. L. Atkin, J. Lehner, Hecke operators on To{m), Math. Ann. 185, (1970), 134-160. 
[AS] M. Akbas, D. Singerman, The normaliser ofTo{N) in PSL2{M), Glasgow Math. J. 32 
(1990), 317-327. 

[C] J. H. Conway, Understanding Groups like ro(Ar), Groups, difference sets and the monster 

(Columbus, Ohio, 1993), Ohio State Univ. Math. Res. Inst. Publ., 4, de Gruyter, BerUn, 
1996, 327-343. 

[CLLT] S. P. Chan, M. L. Lang, C. H. Lim, S. P. Tan, The invariants of the congruence subgroups 
Go{P) of the Hecke group, Illinois J. of Math. 38 (1994), 636 - 652. 

[CN] Conway, J. H., Norton, S. P., Monstrous Moonshine, Bull. London Math. Soc. 11 (1979), 
308-338. 

[La] M. L. Lang, Normalisers of the congruence subgroups of the Hecke groups G4 and Gg, 

I. of Number Theory 90 (2001), 31-43 
[LLT] M. L. Lang, C. H. Lim, S. P. Tan, Independent generators for congruence subgroups of 

Hecke groups, Math. Z. 220 (1995), 569 - 594. 
[LT] M. L. Lang, S. P. Tan, Normalisers of the congruence subgroups of the Hecke group G5 

//, Proc. Amer. Math. Soc. 128 no. 8 (2000), 2271-2280. 
[LI] A. Lcutbcchcr, Uber die Heckeschen Gruppen G{\), Abh. Math. Scm. Hambg. 31 (1967), 

199 - 205. 

[L2] A. Leutbecher, Uber die Heckeschen Gruppen G(A), //, Math. Ann. 211 (1974), 63-68. 
[R] D. Rosen, A class of continued fractions associated with certain properly discontinuous 

group, Duke Math. 21(1954), 549-563. 

DEPARTMENT OF MATHEMATICS, 
NATIONAL UNIVERSITY OF SINGAPORE, 
SINGAPORE 117543, 
REPUBLIC OF SINGAPORE 



e-mail : matlmlSmath . nus . edu . sg 



